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CLAIMS 



[Claim(s)] 

[Claim 1]While an outline of a face and component parts of a face are included, 
template matching which asks for a matching degree of several face templates which 
differ in the direction of a light soaroe irradiated by direction of a l^ind of face and a 
face and face at least, and a face picture containing a person's face is performed, A 
matching method acquiring information about the direotion of a light source irradiated 
by direction of a face and this face in a face area whioh determines a faoe template 
with this highest matching degree, and where tJiis matching degree contains a face on 
said face picture based on the highest face template. 

[Claim 2]The matching method according to claim 1 setting up a field of a prescribed 
range which includes said face area in said face picture as a face area candidate, and 
performing said template matohing in this face area candidate. 
[Claim 3]The matching meUiod according to claim 1 or 2 characterized by coming to 
create said two or more face templates beforehand. 

[Claim 4]It is created when said two or more face templates change direction of a face 
in two or more standard templates beforehand created according to a l<ind of said face, 
and the direotion of a light source, The matching method according to claim 1 or 2 
performing said template matching acquiring a face template by which direction of said 
face and the direction of a light source were changed. 

[Claim 5]A matching method of four given In any 1 paragraph Irom claim 1 performing 
said template matching by asking for a matching degree hierarchical about said two or 
more face templates. 

[Claim 6]Based on information about direction of said face and the direotion of a light 
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source which were acquired from claim 1 by a matching method of five given in any 1 
paragraph. An Image processing method searching for beige oonoentration distribution 
in said face area, defining beige space in said face picture based on this concentration 
distribution, and extracting a field corresponding to said beige space in said face 
picture as said face area. 

[Claim 7]The image processing method aocordingto claim 6 performing predetermined 
image processing to said complaxion range. 
[Claim 8]A matching device comprising; 

While an outline of a face and component parts of a faoe are included, template 
matching which asks for a matching degree of several face templates which differ in 
the dlrectiofi of g light source irradiated by direction of a kind efface and a face and 
face at least, and a face picture containing a person's face is performed, A template 
matching means which determines a f^oe template with this highest matching degree. 
An information acquisition means which acquires information about the direction of a 
li^t source irradiated by direction of a face and this face in a face area where this ' 
matching degree contains a face on said face pioture based on the highest face 
template. 

[Claim 9]T?ie matching device according to claim 8 which is further provided with an 
area setting means which sets up a field of a prescribed range which includes said 
face area in said fiaoe pioture as a face area candidate, and is characterized by said 
template matching means being a means to perform said template matching in said 
fao8 area candidate. 

[Claim 10]The matching device aocordingto claim 8 or 9 having ftjrther a template 
memory measure which memorizes said two or more face templates. 
[Claim 11]A standard template memory measure which memori:ses two or more 
standard templates beforehand created according to a kind of said face, It has further 
a template preparing means which creates said two or more face templates by 
changing direction of a face in said two or more standard templates, and the direction 
of a light source. The matching device according to claim 8 or 9, wherein said template 
matching means is a means to peribrm said template matching acquiring a face 
template from which direction of said face and the direction of a light source which 
were created by this template preparing means differ. 

[Claim 12]A matching device of 1 1 given in any 1 paragraph from claim 8, wherein said 
template matching means is a means to perform said template matching by asking for 
a matching degree hierarchical about said two or more face templates. 
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[Claim 13]An image processing devic© comprising: 

A means to searcli for beige concentration distribution in said face area based on 

information about direction of said face and the direction of a light source wliich were 

acquired from claim 8 by a matching device of 1 2 given in any 1 paragraph. 

A means to define beige space in said face picture based an this concentration 

distribution. 

A means to extract a field corresponding to said beige space in said i^cs picture as 
said face area. 

[Claim 14]The image processing device according to claim 13 having further a means 
to perform predetarmlned image processing to said complexion range. 
[Claim 1 5]A recording medium characterized by comprising the following which 
recorded a program for making a computer perform a matching method and in whioh 
computer reading Is possible. 

While an outline of a face and component parts of a face are inoiuded, template 
matching which asks for a matching degree of several face templates which differ In 
tiie direction of a light source irradiated by direotion of a kind of face and a face and 
face at least and a face picture containing a person's face is performed, A procedure 
of determining a face template with this highest matching degree. 
A procedure which acquires information about the direction of a light source irradiated 
by direotion of a face and this face in a face area where this matching degree contains 
a face on said face picture based on the highest face template. 

[Claim 16] A procedure of having further the procedure of setting up a field of a 
prescribed range which includes said face area in said fece picture as a face area 
candidate, and performing said template matching, A recording medium being the 
procedure of performing said template matching in this fece area candidate and in 
which the computer reading according to claim 15 is possible. 

[Claim 1 7]A recording medium which is characterized by coming to create said two or 
more face templates beforehand and in which the computer reading according to olaim 
15 or 16 is possible. 

[Olaim 18]It has further a procedure created by changing direction of a face in two or 
more standard templates beforehand created in said two or more face templates 
according to a kind of said face, and the direction of a light source, A recording 
medium in which the computer reading according to claim 15 or 16 is possible, wherein 
a procedure of performing said template matching is a procedure of performing said 
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template matching acquiring a faoe template by which direction of said face and the .. 
direction of a light source were changed. 

[Claim 19]A recording medium which can computer read an any 1 paragraph 
statement of 18 from claim 15. wherein a procedure of performing said template 
matching is a procedure of performing said template matching by asking for a 
matching degree hierarohioal about said two or more face templates. 
[Claim 20]A recording medium In which computer reading Is possible, oomprising: 
A procedure of searching for beige concentration distnbution in said face area based 
on information about direction of sgfd face and the direction of a light source which 
were acquired from claim 1 by a matching method of five given in any 1 paragraph. 
A procedure of defining beige space in said face picture based on this oorioentration 
distribution, 

A procedure of extracting a field corresponding to said beige space in said face 
picture as said face area. 

[Claim 21]A recording medium having further the procedure of performing 
predetermined Image processing to said complexion range and In which the computer 
reading according to claim 20 Is possible. 



[Translation donej 
DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the InventionjA matching method and a device which perform template 
matching of the picture and template in which this invention contains a person's face. 
It Is related with the recording medium which recorded the program for mailing a 
computer perform the image processing method, the device, matching method, and 
image processing melJiod using these methods and devices and in which computer 
reading is possible. 
[0002] 

[Description of the Prior* Art]In pictures, such as a photograph obtained by photolnga 
person image using a negative film, a reversal film, etc., the part which attracts 
attention most is a person's face. Therefore, in order to improve iJie vanity of a 
picture, it is necessary to perform image processing to a picture so that it may 
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beooms a color wrth a par-san's proper face, concentration, and gradation. In order to 
perform image processing so that it may become a color with a person's proper face, 
concenlration, and gradation on the other hand, it is necessary to extract a person's 
feoe area from a picture appropriately. For this reason, extract the candidate of a 
complexion rangs out of the picture which includes a person, for example, define a 
beige color space based on the histogram of each candidate's chroma and a 

complexion range is determined, The method (JP,6-309433,A) and two or more face 
templates which make this complexion range a face area are prepared. After 
performing template matching of this face template and picture and extracting a face 
area candidate, extracting a face area from a picture by the method (JP,8-63597,A) of 
extracting a face area based on the beige distribution in a face area candidate etc., 
and performing image processing is proposed, 
[0003] 

[Problem(s) to be Solved by the Invention] However, tha method indicated to 
above-mentioned JP,6-30g433,A extracts the field in the baige space appointed 
beforehand as a complexion range. Therefore, since the beige concentration in a face 
picture continues broadly and is distributed if the contrast of a face picture is large, a 
possibility that the field which has the flesh color which is not contained in the defined 
beige space exists in a face area becomes high, and, as a result, all tha fields of a face 
may be unable to be extracted. When the color of the background of a face is included 
in the defined beige space, a background wiil also be extracted with a face picture. On 
the other hand, the metiiod indicated to above-mentioned JP,8-63597,A, When 
extracting a face area from the face area candidate by whom what can appoint the 
field of a certain amount of face by template matching using a face template was 
extracted based on beige distribution, the same problem as the method indicated to 
above-mentioned JP,6-309433.A arises. 

[0004]This invention is made in \new of the above-mentioned situation, and the 
contrast of the face contained in a face picture is taken into oohsidaration, A 
matching method and a device which can acquire the information for extracting a face 
area appropriately. It aims at providing the recording medium which recorded the 
program for making a computer perform this matching method, the image processing 
method using a device, a device, a matching method, and an image processing method 
and in which computer reading Is possible. 
[0005] 

[Means for Solving the Profalem]While a matching method by this invention contains 
an outline of a face, and component parts of a face. Template matching which asks for 
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a matching degree of several face templates which differ in the direction of a light 
source irradiated by direction of a kind efface and a face and face at least, and a face 
picture containing a person's face is performed, A face template with this highest 
matching degree i$ determined, and information about the direction of a light source 
irradiated by direction of a fece and this face in a fiace area where this matching 
degree contains a i^ce on said fage picture based on tho highest face template is 
acquired. 

[OOOelHere, a "face template" contains component parts of faces, such as an outline 
of a face, eyes, a mouth, a nose> and an ear, about human being's face, and is obtained 
by changing at least the direction of a light source irradiated by directioh of a kind of 
face and a face, and face. In a "face template" in this invention, when direction of a 
face changes, shape of an outline of a face differs from a focating position of 
component parts of a face. When the direction of a light source changes, 
concentration distribution of a field surrounded by outline of a face differs. For 
example, if the direction of a light source is a transverse plane of a face, 
concentration distribution will become uniform, but if the direction of a light source is 
right-hand side toward a template, it will be bright in right-hand side of a field 
surrounded by outline of a face, And left-hand side will serve as dark concentration 
distribution. As this "face template", a face may be expressed in three dimensions or 
it may express in two dimensions. As "a kind of faoe", an adult, a child, a woman, a 
male, yellow-skinned races, the white races, or a black race is mentioned. In addition 
to direction of a kind efface and a face, and the direction of a light source, a ftice 
template from which size of a faos differs may be used. 

[0007] Direction of a face can be known if a position of component parts effaces, such 
as shape of an outline of a face, eyes, and a nose, is known. Therefore, "Information 
about direction of a face" means information about a position of contour shape of a 
face in a face template with the highest matching degree, and component parts of a 
face, 

[0008]When the direction of a lijht source with which a face is irradiated is changed, 
distribution of a luminosity in a face will be changed variously and, as a result, 
oonoentratlon distribution of a face will be changed variously. Therefore, "information 
about the direction of a light source" means Information about concentration 
distribution of a face area in a face template with the highest matching degree. 
[0009]In a matching method by this invention, it is preferred to set up a field of a 
prescribed range which includes said face area in said face picture as a face area 
candidate, and to perform said template matching in this face area candidate. 
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[0010]Her6, "a field of a prescribed range Including a faoe area." may be set up by 
displaying a face picture on a displaying means of a monrtor etc., and surrounding a 
face area in fields, such as a rectangle and a round shape, and may be set up by 
dividing a face picture into two or more fields beforehand, and extracting a field 
including a face area from a divided field. 

[001 1]In a matching method by this invention, said two or mora face templates may be 
created beforehand. 

[0012]It creates by changing direction of a face in two or more standard templates 
beforehand created in said two or more face templates according to a kind of said 
faoe, and the direction of a light source. It may be made to perform said template 
matching, acquiring a faoe template by which direction of this face and the direction of 
a light source were changed, 

[0013]In a matching method by this invention, it is preferred to perform said template 
matching by asking for a matching degree hierarohioal about said two or more face 
templates further again. 

[001 4] Here, "it asks for a matching degree hierarchical" fixes two elements (for 
ei<ample, direction of a faoe and the direction of a light source) among three elements 
of direction of a kind efface and a face, and the direction of a light source, makes one 
element (kind of face) a variable, and means performing template matching by 
changing only one element. [ it ] [ it ] For example, when one element made into a 
variable is made into a kind of fece, a kind efface is determined by performing 
template matching about a kind of faoe first, and asking for a template with the highest 
matching degree. Subsequently, a kind of face and the direction of a face are 
determined by fixing a kind efface, performing template matching by making other one 
element (for example, direotion of a face) into a variable, and asking far a template 
with the highest matching degree. And a kind of fece In a face picture, the direction of 
a face, and the direction of a light souroa are determined by fixing a kind of face, and 
direction of a fBce, performing template matching by making one element (the 
direction of a light source) of further others into a variable, and asking for a template 
with the highest matching degree. 

[0015}Based on information about direction of said faoe and the direotion of a light 
source which were acquired by a matching method by this invention, an image 
processing method by this invention. Beige concentration distribution in said faoe area 
is searched for, beige space in said face picture is defined based on this concentration 
distribution, and a field corresponding to said beige space in said face picture is 
extracted as said face area. 
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[001 6]Hare. saying based on information about direction of a face and the direction 
of a liglrt source, beige concentration distribution in a fao© area is searched for." An 
outline of a face area is presumed from Information about direction of a face, and it 
says acquiring distribution of a high-density area In an outline of a face area, and a 
low-density area from information about a position of component parts of a face in 
infomnatron about direction of a face, and the direction of a light sourco. 
[0017]It says [ "which defines beige space based on concentration distribution" ] 
appointing beige space at all the beigo fields of a high-density area and a low-density 
area which exist all over a face area based on concenl^ation distribution being 
included. 

[0018]In an image processing method by this invention, it is preferred to perform 
predetermined image processing to said complexion range. 
[0019]fHere, as *image processing", processing of versatility, such as a color 
conversion process which considers a complexion range as a request and which is 
changed beige, gray-scale^conversion processing which changes gradation of a face 
area, and sharpness emphasis processing, can be used. 

[0020]This invention is characterized by a matching device comprising the following. 
While an outline of a face and component parts of a face are included, template 
matching which asks for a matching degree of several face templates which differ in 
the direction of a light source jrradiated by direction of a l<ind efface and a face and 
face at least, and a face picture containing a person's face is performed, A template 
matching means which determines a face template with this highest matching degree. 
An information acquisition means which acquires information about the direction of a 
light source irradiated by direction of a face and this face in a face area where this 
matching degree contains a face on said face picture based on the highest face 
template. 

[0021 ]In a matching device by tiiis invention, It has further an area setting means 
which sets up a field of a prescribed range which includes said face area in said face 
picture as a face area candidate, and, as for said template matching means, it is 
preferred that It is a means to perform said template matching in said faoe area 
candidate. 

[0022]It may be made to have further a template memory measure whtch memorizes 
said two or more face templates in a matching device by this invention. 
[0023]In a matching device by this Invention, A standard template memory measure 
which memorizes two or more standard templates beforehand created according to a 
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kind of said facB. It has further a template pr-eparing means which creates said two or 
more face templates by changing direction of a face in said two or more standard 
templates, and the direction of a light source, It is good also as a means to perform 
said template matching, acquiring a face template from which direction of said face 
created by this template praparine means and the direction of said light source differ 
said template matching means. 

[0024]As for said template matching means, in a matching device by this invention, it 
is preferred that it is a means to perform said template matching by asking for a 
matching degree hierarchical about said two or more face templates llirther again. 
[0025]This invention is characterized by an image processing device comprising the 
following. 

A means to search for beige concentration distribution In said face area based on 
information about direction of said face and the direction of a light source which were 
acquired by a matching device by this invention. 

A means to define beige space in said face picture based on this concentration 
distribution. 

A means to extract a field corresponding to said beige space in said face picture as 
said face area. 

[0026]In an image processing device by this invention, it is preferred to have further a 
means to perform predetermined image processing to said complexion range. 
[0027]A matching method and an image processing method by this invention may be 
recorded on a recording medium in which computer reading is possible as a program 
for performing a computer, and may be provided. 
[0028] 

[Effect of the InventionlAccording to the matching method and device by this 
invention, template matching of two or more face templates and a face picture is 
performed. The face area and the face template with the highest matching degree 
which are contained in a face picture are determined, and the information about the 
direction of the light source irradiated by the direction of a face and face in the face 
area on a face picture is acquired based on this face template. The position of the 
shape of the outline of a face and the component parts of a face can be known from 
the information about direction of a face here, and the concentration distribution of 
the face area changed according to the direction of the light source irradiated by the 
face from the information about the direction of a light source can be known. 
Therefore, processing of versatility, such as extraction of a complexion range and 
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extraction of the component parts of a face, can be made easy to perform by using 
the information about direction of a face and the direction of a light source. 
[0029]As compared with the case where template matching is performed in the whole 
face piotwre. the calculation time for matching can be shortened by setting up the field 
of the prescribed range which inoludes a face area In a face picture as a face area 
oandidate, and performing template matching In this face area candidate. 
t0030]Temp[ato matching can be performed only by using eaoh face template one by 
one, without modifying a face template in any way by creating two or more face 
templates beforehand. 

[0031]By performing template matching, creating two or more face templates and 
changing direction of a face and the direction of a light source by changing further 
again direction of the face in the standard template created beforehand, and the 
direction of a light source, The number of the templates to save can be lessened and 
the capacity of the memory measure which memorizes a template by this can be 
reduced. 

[0032]Sinoe it becomes unnecessary to perform template matching about no face 
templates by asking for a matching degree hierarchical, the oparaUon for template 
matchrng can be performed at high speed. 

t0033]ln the image processing method and device by this invention, the beige 
conoontration distribution in tiie face area in a -feo© pioUire is searched for based on 
the information about the direction of a fgoe and the direction of a light source which 
were extracted by the matching method and device by this invention. That is. the 
outline of a face area Is presumed from the information about direction of a face, 
distribution of the high-density area in the outline of a face area and a low^density 
area is acquired, and this is obtained from the information about the position of the 
component parts effaces, such as a nose in the information about direction of a face, 
and the direction of a light source as beige concentration distribution. It is the 
information as which a high-density area and a low-density area express how it is 
distributed and the contrast of the field of a face is formed, and beige space is defined 
as including all the beigs fields of the high-density area and low-density area which 
exist all over a face area based on this concentration distribution. [ in / In this 
concentration distribution / a face area ] For this reason, even if the contrast of a 
face area is large, beige space can be defined as extracting all Wie flesh colors of a 
face area, and thereby, a complexion range can be correctly extracted from a f^oe 
picture. 

[0034]Image processing which considers the face area !n a face picture as a request 
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should be performed by performing image processing to the extracted complexion 

range. 

[0035] 

[Embodiment of the Invention] With reference to drawings, the embodiment of this 
invention is described below. 

rOOSel Drawing 1 is a schematic block diagram showing the composition of the image 
processing device which applied the matching device by the embodiment of this 
invention. As shown in drawing 1 , the image processing device by this embodiment is 
provided with the following. 

It is what processes to ih& image data SO showing the picture containing a person's 
face which was picturized with the digital camera or was read with the scanner, and 
obtains tiie processed imago data SI, The reading means 1 which reads the image 
data SO which reads the image data SO from the media on which the Image data SO 
was recorded, or was transmitted via the network. 

A face extraction means 2 to extract a person's face from the picture expressed by 
the read image data SO. 

The image processing means 3 which performs image processing to the irnage data SO, 
and obtains the processed image data S1. 

The input means 6 which consists of the output means 4 which outputs the processed 
image data SI to a printer, a network, a recording medium, a monitor, etc, the monitor 
5 which displays image data SO grade, a keyboard for performing various inputs to the 
face extraction means 2, the image processing means 3, and the output means 4, a 
mouse, etc. 

[0037]The face extraction means 2 is provided with the following. 

A territorial extension means 1 1 to specify the face area candidate F who mentions 

later in the picture expressed by the image data SO. 

The area extraction means 1 2 which extracts the face area candidate F specified in 
the territorial extension means 11. 

The template memory measure 13 which memorized template data TD showing two or 
mora templates T, 

The matching means 14 which performs template matching with the face area 
candidate F extracted in the template T and the area extraction means 12, and 
searches for the information I about direction of the face in the face area candidate F, 
and the direction of a light source, A complexion range extraction means 1 5 to extract 
the complexion range H fi-om the face area candidate F based on the information 1 
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searchad for in the matching means 14, 

[0038]The territorial extension means 11 specifies the field which contains a face as 
shown, for example In drawing 2 as the face area candidate F with the directions from 
tile input means 6 to the piotur© (it shall express with SO also about a picture below) 
SO which displayed the image data SO on the monitor 5. and waa displayed on the 
monitor 5. It may not be limited to a rectangle, and the face area candidate F may be 
circular, or may be an ellipse form. The number which divides the picture SO into two 
or more fields, and specifies each field may be given, and the face area candidate F 
may be specified by specifying the number of the field containing a face. For example, 
as shown in drawing 3 . when the picture SO is divided into the field of 5x5, the faoe 
area candidate's F number is set to 22. 

[0039]lt may add to the image data SO beforehand by making into tag information the 
face area candidate's F information (for example, a face area candidate's position, the 
number of the field containing a face) whioh sets up the fees area candidate F 
boforehand when the image data SO is acquired, and was set up. TTiereby, in the 
territorial extension means 11. even if there are no directions from the input means 6. 
based on the tag information of the Image data SO. the picture SO as which the face 
area candidate F was specified can be displayed on the monitor 5. 
[0040]Although the area extraction means 12 extracts the face area candidate F 
specified in the territorial extension means 11, it may change some the face area 
candidate's F sizes specified in the territorial extension means 1 1 In the case of 
extraction so that fields other than a face area may decrease as much as possible in 
the face area candidate F. Although the extracted face area candidate F may be 
displayed on the monitor 5 by extracting the face area candidate F in the area 
extraction means 1 2. as shown in drawing 2 , the face area candidates F boundary may 
be displayed on the picture SO. 

[004l!]The size and template data TD showing several templates T which shall be 
different in the direction of the li^t source which It is suitable and is irradiated are 
created and memorized with an on-the-spot photo or computer graphics about two or 
more kinds effaces by tiie template memory measure 13. It is more desirable in order 
that the template T may not receive the influence by blur at the time of a noise or 
photography, if template data TD is created with computer graphics here. The 
component parts effaces, such as an outline of a face and eyes, a nose, and a 
supercilium, according to the kind efface, a size, and direction are contained in the 
template T. 
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[0042]l-lara, as a kind of face, an adult, a child, a baby, an old man, a woman, a male, 
yeiiovr-aldnnQd races, tfie white races, or a black race is mentioned. In order to differ 
direction of a face, a face makes 0 times tlie state where it turned to the transverse 
plane, changes the outline of a face, and arrangement of component parts according 
to direction of a fiaoa which is different every 15 ds^ees to 0 to 360 degrees, and 
should just consider it as the template T, for example. In order to differ the direction 
of a fight source, a light source makes 0 times the state of being in the transverse 
plane of a face, and should just make the template T the picture of the face in the 
state whore it irradiated with light towards the face from a direction which is different 
every 15 degrees in **90 degrees about a tota! of four directions of four directions 
and an oblique direction. Template data TD may be created without taking the size of 
a face into consideration. In this case, in the matching means 14 mentioned later, tiie 
face area candidate F and the template T will be normalized, and matching will be 
performed, 

[0043]As for template data TD. It is preferred to hierarchize and to memorize to the 
template memory measure 13. For example, it is prefarrad to match with the hierarchy 
of the low rank a baby's face the face of the baby from whom direction of a face 
differs, and to matoh with the hierarchy of the low rank in direction of the face of 
further each the face of the baby from whom the position of the shadow on a face, i.e.. 
the position of a light source, differs, as shown in drawing 4 . Thus, after the kind of 
face is first determined by hierarchizing and memorizing template data TD, template 
matching can be performed hierarchical so that the direction of a face according to 
the kind may be determined and the direction of the source of crepuscular rays may 
be determined ftjrther, Therefore, as compared with the case where template 
matching is performed using all the templates T, template matching of the face area 
candidate F and the template T in the matching means 14 can be parformod efficiently. 
[0044]The on-the-spot photo or the three-dimensional geometric model of computer 
graphics showing two or more kinds effaces may be memorized as a template data TD. 
Drawing 5 i s a figure showing templatte data TD showing ihe three-dimensional 
geometric model of a male face. Thus, when template ddta TD is made into a 
three-dimensional geometric model. In the matching means 4 which changes and 
mentions later various the directions of the light source which displays a 
three-dimensional geometric model on the monitor 5, and in which the input means 6 
is suitable and a face is irradiated, template matching with the face area candidate F 
can be performed. Thus, by making template data TD into a three-dimensional 
geometric model, changing direction of the face in the template T expressed by this 
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template data TD, and the direction of a light source, and performing template 
matching with the face area candidate F, Template data TD memorized to the 
template memory measure 13 can be lessened, and, thereby, the capacity of the 
template memory measure 1 3 can be reduced. 

[0045]ThQ direction of the light source of a face whfch ft fs suitable and is Irradiated 
oan bs changed hierarchical also in this case, template matching can be performed, 
and, thereby, template matching with the template T can be efficiently performed with 
the face area candidate F. 

[0046]The template T binary^ized as a template data TD may be expressed. 
[00473lf the data showing the face area candidate F extracted in the area extraction 
means 12 is inputted, the matching means 14 will begin to raad template data TD one 
by one from the template memory measure 13, and will perform template matching 
with the template T with the face area candidate F. The face area candidate F and the 
template T are binary-ized, and, spscifically, it asks for a matching degree with the 
face area candidate F and the template T which were binary-ized. When the template 
T is binary-ized, the template T is used as it is. And a matching degree asl<s as the 
Information I showing direction of a face [ In / for dirxaction of the face in the template 
T used as the maximum and the direction of a light souroe / the face area candidate 
F ], and the direction of a light source. If F' and the binary-ized data of the template T 
are specifically made into TD' for the data in the face area candidate F who 
binary-ized, whenever it ohgnges the position and kind of template T, as shown in the 
following formula (1), matohfng degree M of data F and data TD' will be computed, 
[0048] 

M=(sigma!Fu '"TDu'|)/n (1) 

however, the pixel number of the picture expressed by pixel value n:data Fij' or data 
TDU" of the picture expressed by pixel value TDU'idata F' of the picture expressed by 
Fij':data F', and corresponding data TD' — and. It asks as the information I as which 
computed matching degree M expresses direction of a face [ in / for direction of the 
face in the template T used as the maximum and the direction of a light souroe / the 
face area candidate F ], and the direction of a light source. Template matching may be 
performed without blnary-lzing the template T and the face area candidate F. 
[0049]Har8, direction of a face oan be known if the position of the component parts of 
faces, such as shape of the outline of a face. eyes, and a nose, is known. Therefore, 
the information about direction of a face means the information about the position of 
the component parts of faces, such as contour shape of the fece in the template T 
with the highest matching degree, eyes, and a nose. 
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[0050]When the direction of the light souroa with which a face is irradiated is changed, 
distribution of the luminosity in a face will be changed variously and. as a resutt, the 
oonoentration distribution of a face will be changed variously. Therefore, the 
information about the direotion of a light source means tha information about the 
concentration distribution of tha face in the template T with the highest matching 
degree. 

[0051]As template data TD shows drawing 4 t he template memory measure 13, when 
hierarchizing, after matching of the kind of face, What is necessary is to match only 
direction of a face in the low order hierarchy of the face of the matched kind, and after 
matching of direction of a face should match only about the direction of a light source 
in tha low order hierarchy of direction of a face further. 

[0O52]By inputting beforehand the kind (for example, it is by a baby's face which Is a 
female face) of fece contained in the picture SO at the time of the territorial extension 
in the territorial extension means 11. In the matching means 14, iJie template T 
showing the inputted face of a kind can be preferentially used for matching, and this 
oan also perform template matching efficiently. 

[0053]Lag-Poiar conversion of the face area candidate F and the template T may be 
carried out, and template matching may be performed. Since human being's face is 
making circularly near shape, while it makes the center of a face picture a pole, here 
the face picture with which light was irradiated from the right 90 degreQ direction as 
shown, for example in drawing 6 , If Log-Polar conversion is carried out on the basis of 
the Y-axis in drawing 6 . on polar coordinates, the outline of a face will extend to an 
abbreviated horizontal direction, as shown in drawing 7 . Therefore, template matching 
can be made easy to perform as compared with the case where template matching of 
the fields which have circularly near shape by carrying out Log-Polar conversion of 
the face area candidate F and the template T Is carried out. 
[0054]Based on tiie information 1 showing the direction and the direction of a li#it 
source of a face in the face area oendidate F obtained In the matching means 14, the 
complexion range extraction means 15, Area division of the Ibco area candidate F Is 
carried out to a bright field, l.o., a low-density area, and a dark field. i.B., a high-density 
area, beige space is defined for every field, and a complexion range is extracted. 
Extraction of this complexion range H is performed as follows. First, the outline of the 
face in the faca area candidate F oan be known from the information about direction 
of a face, and it can know, distribution, i;©.. the concentration distribution, of the bright 
field in the field surrounded by the outline from the infarmation which expresses the 
direction of a light source further, and a dark field. Although the position of the 
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component parts of a face ogn be known from the information about direction of a 
face in this case, based on the information especially about the position of a nose and 
eyes, the position of the oheek under the nose and eyes which become brighter than 
other fields in a face can be known. And beige space is defined based on ths position 
of concentration distribution and a nose, and a cheek, and the complexion range H Is 
extracted. 

[0055]As specifioally shown in drawing 6 . when light is Irradiated from the right-hand 
side of the face which turned to the transverse plane, although it is bright In the fiold 
in the right half of a face (it is considered as a right area below), the field (it is 
considered as a left area below) of a left half will become dark. Since the nose and the 
cheek have projected in human being's face, in a right area, the right-hand side and 
the cheek of a nose become brighter than other fields. Also in a loft area, the portions 
of a nose and a cheek become brighter than other fields. 

[0056]Therefore, about a right area, beige space is defined as including the brightest 
portion, i.e., the rieht-hand side of a nose and the portion of a cheek, and the field in 
the right half of a face Is extracted. On the other hand, beige space is defined as 
Including the left-hand side of a nose, and the portion of a cheek also about the left 
half of a face, and the field in the left half of a face is extracted. 
[0057]The image processing means 3 performs image processing, such as gradation 
processing, a color conversion process, a sharpness process, to the image data SO of 
this complexion range H, and obtains the processed image data S1 so that it may 
become the concentration and the color whioh the extracted complexion range H 
considers as a request. Image processing may be performed about fields other than 
the complexion range H. Here, as a sharpness process, according to the size of the 
- compJexion range H, i.e., the size of a face, when a face is large, the grade of 
sharpness emphasis can be weakened, and it can process so that the grade of 
sharpness emphasis may be enlarged, when a face is small. 

[0058]Subsequenitly, operation of this embodiment is explained. Drawing 8 is a flow 
chart which shows operation of this embodiment. First, tiie image data SO Is read in 
the reading means 1 (Step S1), and it is inputted into the face area extraction means 
2. The image data SO is displayed on the monitor 5 (Step S2), and the face area 
candidate F Is specified by the input from the input means 6 (Step S3). And the face 
area candidate F Is extracted !n the area extraction means 12 (step S4). The 
extracted face area candidate F is inputted into the matching means 14, and template 
matching of the template T and the face area candidate F to whom it is expressed by 
template data TD is performed using template data TD memorized by the template 
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memoiY measure 13 here CStep S5). And an end of tsmplate matching will generate 
the information I which expresses direction of a face and the direction of a light 
source based on the template T with the largest matching degree (Step S6). 
[0059]Based on the information I, the complexion range extraction means 15 extracts 
the complexion range H from the face area candidate F, as mentioned above (Step S7). 
And image processing, such as gradation processing, a color conversion process, and 
a sharpness process, is performed to the complexion range H in the picture expressed 
by the image data SO in the image processing means 3, and the processed image data 
81 is obtained (Step S8), And the processed Image data Si is outputted In the output 
means 4 (step S9), and ends processing. 

[0060]Having acquired the information I which carries out template matching of the 
face area candidate F by two or more templates T, and expresses direction of the face 
in the face area candidate F, and the direction of a light source in tiiis embodiment 
Thus, a sake, The position of the oontour shape of a face and the component parts of 
a face can be known from the information about direction of a face, and the 
concentration distribution of the face area candidate F changed according to the 
direction of the light irradiated by the face from the information about the direction of 
a light source can be known. And among the information 1, the outline of a face area is 
presumed from the information about direction of a face, and the concentration 
distribution in the outline of a face is acquired from the information about the position 
of the component parts of faces, such as a nose in the information about direction of 
a face, and the direction of a light source. Therefore, by defining beige space as 
including all the beige fields of the high-density area and low-density area which exist 
all over the fietd In the outline of the face in the face area candidate F based on this 
concentration distribution. Beige space can be defined as extracting all the faoe area 
candidate's F flesh colors as the contrast of a feco area is large, and, thereby, the 
complexion range H can be extracted correctly. j 
[0061]That the field of the prescribed range which contains a face in the territorial 
extension means 11 is set up as the face area candidate F. and it was made to 
perform template matching in this set-up face area candidate F A sake. The 
calculation time for matching can be shortened as compared with the case where 
template matching is performed in the whole faoe picture. 

[0082] Although the information about direction of a faoe and the information about 
the direction of a light source are acquired as the information I and the complexion 
range H is extracted in the above-mentioned embodiment based on these. Based on 
the information I, the position of the component parts of faces, such as eyes and a 
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noso, oan be known from the face area candidate F, and, thereby, the component 
parts of the face from the face area candidate F can be simply extracted using the 
information I. 



[Translation donej 
TECHNICAL FIELD 



[Field of the InventionjA matching method and a device which perform template 
matching of the picture and template in which this invention contains a person's face. 
It is related with the recording medium which recorded the program for making a 
computer perform the image processing method, the device, matching method, and 
image processing method using these methods and devices and in which computer 
reading is possible. 



PRIOR ART 



[Description of the Prior ArtGin pictures, such as a photograph obtained by photoing a 
person image using a negative film, a reversal film, etc., the part which attracts 
attention most is a person's face. Therefore, in ofder to improve the vanity of a 
picture, it is necessary to perform image processing to a picture so that It may 
become a color with a person's proper faoe. concentration, and gradation. In order to 
perform image processing so that it may become a color with a person's proper faoe. 
concentration, and gradation on the other hand, it is necessary to extract a person's 
face area from a pictare appropriately. For this reason, extract the candidate of a 
complexion range out of the picture which includes a person, for example, define a 
befgB color space based on the histogram of each candidate's chroma and a 

oomplexion range is determined, The method CJP,6-309433,A) and two or more face ' 
templates which mal<e this complexion range a face area are prepared, After 
performing template matching of this face template and picture and extracting a face' 
area candidate, exiracting a face area from a picture by the method (JP,8-63597,A) of 
extracting a face area based on the beige distribution In a face area candidate etc., 
and performing image processing is proposed. 
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[Translation done.] 

EFFECT OF THE INVENTION 



[Effect of the Invention]AcGording to the matching method and device by this 
invention, template matching of two or more face templates and a face picture Is 
performed. The face area and the face template with the highest matching degree 
which are contained in a faoe picture are determined, and the information about the 
direction of the light source irradiated by the direction of a face and face in the face 
area on a face picture is acquired based on this face template. The position of the 
shape of the outline of a face and the component parts of a faoe can be known from 
the information about direction of a faoe here, and the concenlration distribution of 
the face area changed according to the direction of the light source irradiated by the 
faoa from the information about the direction of a light source can be known. 
Therefore, processing of versatility, such as extraction of a complexion range and 
extraction of the component parts of a face, can be made easy to perform by using 
the information about direction of a faoe and the direction of a light source. 
[0029]As compared with the case where template matching is performed in the whole 
face picture, the calculation time for matching can be shortened by setting up the field 
of the prescribed range which includes a face area in a face picture as a face area 
candidate, and performing template matching in this face area candidate. 
[0030]Templat6 matching can bo performed only by using each faoe template one by 
one, without modifying a face template in any way by creating two or more faoe 
templates beforehand, 

[0031]By performing template matching, creating two or more face templates and 
changing direction of a face and the direction of a light source by changing fiirther 
again direction of the faoe in the sitandard template created beforehand, and the 
direction of a light source, The number of the templates to save can be lessened and 
the capacity of the memory measure which memorizes a template by this oan be 
reduced. 

t0032]Since it becomes unnecessary to perform template matching about no face 
templates by asking for a matching degree hierarchical, the operation for template 
matching can be performed at high speed. 

[0033]In the image processing method and device by this invention, the beige 
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concentration distribution in the face area in a face picture is searched for based on 
the information about the direction of a face and the direction of a light source which 
were extracted by the matohinff method and device by this invention. That is. the 
outline of g faoe area is presumed from the information about direction of a face, 
distribution of the high-density area in the outline of a fiaoe area and a fow-density 
area is acquired, and this is obtained from the I'nfbrmation about the position of the 
component parts offices, such as a nose in the information about dirsotion of a fece, 
and the direction of a fight source as beige concentration distribution. This 
concentration distribution is the information showing how for the high-density area 
and low-density area in a face area to be distributed, and to form the contrast of the 
field of a face. 

Beige space is definsd as including all the beige fields of the high-density area and 
low-density area which exist all over a faoe area based on this concentration 
distribution. 

For this reason, even if the contrast of a face area is large, beige space can be defined 
as extracting all the flesh colors of a face area, and thereby, a complexion range can 
be corraotly extracted from a face picture. 

[0034]lmage processing which considers the face area in a face picture as a request 
should be performed by perfonming image processing to the extracted complexion 
range. 
t0035] 

[Embodiment of the Invention] WiWi reference to drawings, the embodiment of this 
invention is described below. 

[0036] Prawing 1_ is a schematic block diagram showing the composition of the image 
processing device which applied the matching device by the embodiment of this . 
invention. As shown in drawing 1 . the image processing device by this embodiment 
processes to the image data SO showing the picture containing a person's face which 
was picturized with the digital camera or was read with the scanner, and obtains the 
processed Image data SI. 

The reading means 1 which reads the image data SO whioh reads the image data SO 
from tiie media on which the fma^ data SO was recorded, or was transmitted via the 
network. A faoe ©xtraotion means 2 to extract a parson's fece from the picture 
expressed by the read image data SO. The image processing means 3 which performs 
image processing to the image data SO. and obtains the processed image data SI, The 
output means 4 which outputs the processed image data S1 to a printer, a networit, a 
recording medium, a monitor, etc., It has the input means 6 whioh consists of the 
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monitor 5 which displays image data SO grade, a keyboard for performing various 
inputs to the face extraction means 2, the image processing means 3, and the output 
means 4. a mouse, etc. 

[0037JA territorial extension means 1 1 to specify the face area candidate F who 
mentions the face extraction means 2 later in the picture expressed by the Image data 
SO. The area extraction means 12 which extracts the face area candidate F specified 
in the territorial extension means 11. The template memory measure 13 which 
memorized template data TD showing two or more templates T, The matching means 
14 which performs template matching with the face area candidate F extracted in the 
template T and the area extraction means 12, and searches for the information I about 
direction of the face in the face area candidate F, and the direction of a light source. 
Based on the information I searched for in the matching means 1 4, it has a complexion 
range extraction means 1 5 to extract the complexion range H from the face area 
candidate F. 

[0038]The territorial extension means 1 1 specifies the field which contains a fece as 
shown, for example in drawing 2 as the face area candidate F with the directions from 
the input means 6 to the picture (it shall express with SO also about a picture below) 
SO which displayed the image data SO on the monitor 5, and was displayed on the 
monitor 5, It may not be limited to a rectangle, and the face area candidate F may be 
circular, or may be an ellipse form, The number which divides the picture SO into two 
or mere fields, and specifies each field may be given, and the face area candidate F 
may be specified by specifying the number of the field containing a face. For example, 
as shown in drawing 3 , when the picture SO is divided into the field of 5x5, the face 
area candidate's F number is set to 22. 

[0039]It may add to the image data SO beforehand by mal<ing into tag information the 
face area candidate's F infonmation (for example, a face area candidate's position, the 
number of the field containing a face) which sets up the fiaoe area candidate F 
beforehand when the Image data SO is acquired, and was set up. Thereby, in the 
territorial extension means 11, even if there are no directions from the input means 6. 
based on the tag information of the image data SO. the picture SO as which the fece 
area candidate F was specified can be displayed on the monitor 5. 
[0040]Although the area extraction means 12 extracts the face area candidate F 
specified in the territorial extension means 11, it may change some the face area 
candidate's F sizes specified in the territorial extension means 1 1 in the case of 
extraction so that fields other than a face area may decrease as much as possible in 
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the face area candidat© F. Although the extracted face area candidate F may be 
dfsplayed on the monitor 5 by extracting the face area candidate F in the area 
extraction means 1 2, as shown in drawing 2. the face area candidate's F boundary may 
be displayed on the picture SO. 

[0041]The size and template data TD showing several templates T which shall be' 
different in the direction of the light source which it is suiUble and is Irradiated are 
creatod and memorized with an on-the-spot photo or computer graphics about two or 
more kinds of faces by the template memory measure 1 3. It is more desirable in order 
that the template T may not receive the Influence by blur at the time of a noise or 
photography, if template data TD is created with computer graphics here. Tho 
component parts of faces, such as an outline of a face and eyes, a nose, and a. 
supercilium, according to the kind of face, a size, and direction are contained in the 
template T. 

[0042]H6re. as a kind of face, an adult, a child, a baby, an old man, a woman, a male, 
yellow-skinned races, the white races, or a black race is mentioned. In order to differ 
direction of a face, a face makes 0 times the state where it turned to the transverse 
plane, changes the outiine of a face, and arrangement of component parts according 
to diraction of a face which is different every 15 degrees to 0 to 360 degrees, and 
should just consider it as the template T, for example. In order to differ the direction 
of a light source, a light source makes 0 times the state of being In the transverse 
plane of a face, and should just make ihe template T the picture of the face In tJie 
state where it irradiated with light towards tho face from a direction which is different 
every 16 degrees in f<*90 degrees about a total of four directions of four directions 
and an oblique direction. Template data TD may be created without taking the sizfe of 
a face into consideration. In tills case, in the matching means 14 mentioned later, the 
face area candidate F and the template T will be nomialized, and matching will be 
performed. 

[0043]As for template data TD. it is preferred to hierarchize and to memorize to the 
template memory measure 13, For example, it is preferred to match with the hierarchy 
of the low rank a baby's face the face of the baby from whom direction of a face 
differs, and to match with the hierarchy of the low rank in direction of the face of 
ftirther each the face of the baby from whom the position of the shadow on a face, i.e., 
the position of a light source, differs, as shown in drawing 4. Thus, after the kind of 
face is first determined by hierarchizing and memorizing template data TD, template 
matching can be performed hierarchical so that the direction of a feoe according to 
the kind may be determined and the direction of the source of crepuscular rays may 
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be datamilned further. Therefore, as compared wtth the case where template 
matohing is performed using all the templates T, template matching of the fece area 
candidate F and the template T in the matching means 14 can be performed afficientiy. 
[0044]The on-the-spot photo or the three~dimensional gaometric model of computer 
graphics showing two or more kinds effaces may be memorized as a template data TD. 
Drawing!; 5 is a figure showing template data TD showing the three-dimensional 
geometric model of a male face. Thus, when template data TD is made into a 
three-dimensional geometric model, In the matching means 4 which changes and 
mentions later various the directions of the light source which displays a 
three-dimensional geometric model on the monitor S, and in which the input means 6 
is suitable and a face is irradiated, template matching with the feoe area candidate F 
can be performed. Thus, by making template data TD into a three-dimensional 
geometric model, changing direction of the face in the template T expressed by this 
template data TD, and the direction of a light souroa, and performing template 
matohing with the face area candidate F, Template data TD memorized to the 
template memory measure 13 oan be lessened, and. thereby, the capacity of the 
template memory measure 13 can be reduced. 

[0045]The direction of the light source of a face which it is suitable and Is irradiated 
can be changed hierarchical also In this case, template matching can be performed, 
and, thereby, template matohing with the template T can be efficiently performed with 
the face area candidate F. 

[0048]The template T binary-ized as a template data TD may be expressed. 
[0047]If the data showing iJie face area candidate F extracted in the area extraction 
means 12 is inputted, the matching means 14 will begin to read template data TD one 
by one from the template memory measure 13, and will perfomi template matching 
with the template T with the face area candidate F. The face area candidate F and the 
template T are binary-lzed, and. specifioaliy. it asks for a matohing degree with the 
face area candidate F and the template T which were binary-ized. When the template 
T is binary-ized, the template T Is used as it is. And a matohing degree asks as th& 
information I showing direction of a face [ in / for direction of the face in the template 
T used as the maximum and the direction of a light source / the faoe area candidate 
F ]. and the direction of a light source. If F' and the binary-ized data of the template T 
are specifically made into TD' for the data in the face area candidate F who 
binary-ized. whenever It changes the position and kind of template T, as shown in the- 
following formula (1). matching degree M of data F' and data TD' will be computed. 
[0048] 
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M=(sigma|Fij'-TDu"|)/nC1) 

however, the pixel number of the pioture expressed by pixel value n:data Fu' or data 
TDjj' of the picture expressed by pixel value TDij':data F of the picture expressed by 
Fij':data F', and corresponding datg TD' — and. It asks as tha information I as which 
computed matching degree M expresses direotfon of a face [ in / for direction of the 
feoe in the template T used as the maximum and the direction of a light source / the 
face area candidate F ], and the direction of a light source. Template matching may be 
performed without binary-izingthe template T and the face area candidate P. 
[0049]Here, direction of a face can be known if the position of^he component parts of 
faces, such as shape of the outline of a face, eyes, and a nose, is known. Therefore, 
the information about direction of a face means the information about the position of 
the component parts of faces, such as contour shape of the face in the template T 
with the highest matching degree, eyes, and a nose. 

[0050]When the direotion of the iight source with which a face is irradiated is ohanged. 
distribution of the luminosity in a face will be changed variously and, as a result, the 
concontfation distribution of a feoe will be changed variously. Therefore, the 
Information about the direction of a iight source means the information about the 
concentration distribution of the face' in the template T with the highest matching 
degree. 

[0061]As template data TD shows drawing 4 t he template memory measure 13. when 
hierarchizing, after matching of the kind of face. What Is necessary is to match only 
direotion of a face in the low order hierarchy of the face of tiia matched kind, and after 
matching of direction of a feoe should match only about the direction of a light source 
in the low order hierarchy of direction of a face further. 

[0052]By. inputting beforehand the kind (for example, it is by a baby's face which is a 
female face) of face contained in the picture SO at the time of the territorial extension 
in the territorial extension means 11, In the matching means 14, the template T 
showing the inputted face of a kind can be preferentially used for matching, and this 
can also perform template matching efficiently. 

[0053]Log-Polar conversion of the face area candidate F and the template T may be 
carried out, and template matching may be performed. Since human being's face is 
making circularly near shape, while it makes the center of a face picture a pole, here 
tha face pioture with which iight was irradiated from the right 90 degree direction as 
shown, for example in drawing 6. If Log-Polar conversion is carried out on the basis of 
the Y-axis in drawing 6 , on polar coordinates, the outline of a face will extend to an 
abbreviated horizontal direction, as shown in drawing 7 . Therefore, template matching 
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can be made easy to perform as compared with the case where template matching of 
the fields which have circularly near shape by carrying out Log-Polar conversion of 
the face area candidate F and the template T is carried out. 
[0054]Based on the information 1 showing the direction and the direction of a light 
source of a face in the face area candidate F obtained in the matching means 14, the 
complexion range extraction means 15, Area division of the face area candidate F is 
carried out to a bright field, i.e., a low-density area, and a dark field, i.Q., a high-density 
area, beige space is defined for every field, and a complexion range is extracted. 
Extraction of this complexion range H is performed as follows. First, the outline of the 
face in the face area candidate F oan be known from the information about direction 
of a face, and it can know, distribution, i.e., the concentration distribution, of the bright 
field in the field surrounded by the outline from the information which expresses the 
direction of a light source further, and a dark field. Although the position of the 
component parts of a face oan be known from the information about direction of a 
face in this case, based on the information especially about the position of a nose and 
eyes, the position of the cheek under the nose and eyes which become brighter than 
other fields In a 1^ce can be known. And beige space Is defined based on the position 
of concentration distribution and a nose, and a cheek, and the complexion range H is 
extracted. 

[0055]As speofficaliy shown in drawing 6 , when light is irradiated from the right-hand 
side of the face which turned to the transverse plane, although it is bright in the field 
in the right half of a face (it is considered as g right area below), the field (it is 
considered as a left area below) of a left half will bsoome dark. Since the nose and the 
cheak have projected in human being's face, In a right area, the right-hand side and 
the cheek of a nose become brighter than other fields. Also in a left area, the portions 
of a nose and a cheek become brighter than other fields, 

[0056] Therefore, about a right area, beige space is defined as including the brightest 
portion. i.e., the right-hand side of a nose and the portion of a cheek, and the field in 
the right half of a face is extracted. On the other hand, beige space is defined as 
including the left-hand side of a nose, and the portion of a cheek also about the left 
half of a face, and the field in the left half of a face is extracted. 
[0057]The image processing means 3 performs image processing, such as gradation 
processing, a color conversion process, a sharpness process, to the image data SO of 
this complexion range H, and obtains the processed image data Si so that it may 
became the concentration and the color which the extracted compfexion range H 
considers as a request. Image processing may be performed about fields other than 
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the complexion range H. Here, as a sharpness process, according to the size of the 
oomplexion range H, i.e., the size of a face, when a face is large, the grade of 
sharpness emphasis can be weakened, and it can process so that the grade of 
sharpness emphasis may be enlarged, when a face is small. 

[0058]Subsequently. operation of this embodimont is explained. Drawing 8 is a flow 
chart which shows operation of this embodiment. First, the image data SO is read in 
the reading means 1 (Step SI), and it is inputted into the face area extraction means 
2. The image data SO fs displayed on the monitor 5 (Step S2), and the face area 
candidate F is specified by the input from the input means 6 (Step $3). And the face 
area candidate F is extracted in the area extraction means 12 (step S4). The 
extracted face area candidate F is inputted into the matching means 14, and template 
matching of the template T and the face area candidate F to whom it (s expressed by 
template data TD is performed using template data TD memorized by the template 
memory measure 13 here (Step 35). And an end of template matching will generate 
the Information I which expresses direction of a face and the direction of a light 
source based on the template T with the largest matching degree (Step S6). 
[0059]Based on the information I, the oomplexion range extraction means 1 6 extracts 
the complexion range H from the face area candidate F. as mentioned above (Step S7). 
And image processing, such as gradation processing, a color conversion process, and 
a sharpness process, is performed to the oomplexion range H in the picture expressed 
by the image data SO in the image processing means 3, and the processed image data 
SI is obtained (Step S8). And the processed image data SI is outputted in the output 
means 4 (step S9), and ends processing. 

[0060]Having acquired the information I which carries out template matching of the 
face area candidate F by two or more templates T, and expresses direction of the face 
in the face area candidate F, and the direction of a light source in this embodiment 
Thus, a sake, The position of the contour shape of a face and the component parts of 
a face can bo known from the Information about direction of a face, and the 
concentration distribution of the face area candidate F changed according to the 
direction of the light irradiated by the face from the Information about the direction of 
a light source can be known. And among the information I, the outline of a faoe area Is 
presumed from the Information about direction of a ftice, and the conoentration 
distribution in the outline of a face is acquired from the information about the position 
of the component parts of fgoes, such as a nose in the information about direction of 
a face, and the direction of a light source. Therefore, by defining beige space as 
including all the beige fields of the high-density area and low-density area which exist 
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all over the field in the outline of the face in the face area candidate F based on this 
concentration distribution, Beige space can be defined as extracting all the face area 
candidate's F flesh colors as the contrast of a face area is large, and, thereby, the 
compjexion range H can be extracted correctly. 

[006l3"niat the field of the prescribed range which contains a face in the territorial 
extension means 1 1 is set up as the face area candidate F, and it was made to 
perform template matching in this set-up face area candidate F A sake, The 
calculation time for matching can be shortened as compared with the case where 
template matching is performed in the whole face picture. 

[0062]AIthough the information about direction of a face and the information about 
the direction of a light source are acquired as the information I and the complexion 
range H is extracted in the above-mentioned ambodimont based on these. Based on 
the information I, the position of the component parts of laces, such as eyes and a 
nose, can be known from the face area candidate F, and, thereby, the oomponent 
parts of the face from the feoe area candidate F can be simply extracted using the 
information I. 



[Translation done.] 
TECHNICAL PROBLEM 



[ProblemCs) to be Solved by the Inventlon]However, the method indicated to 
above-mentioned JP,6-309433,A extracts the field in the beige space appointed 
beforehand as a complexion range. Therefore, since the beige concentration in a face 
picture continues broadly and is distributed if the contrast of afiace picture is large, a 
possibility that the field which has the flesh color which is not contained In the defined 
beige space exists in a face area becomes high, and,- as a result, all the fields of a faoe 
may be unable to be extracted. When the color of the background of a faoe is included 
In the defined beige space, a background will also be extracted with a face picture. On 
the other hand, the method indioatad to above-mentioned JP,8-63507,A, When 
extracting a face area from the face area candidate by whom what can appoint the 
field of a certain amount of face by template matching using a faoe template was 
extracted based on beige distribution, the same problem as the method indicated to 
above-mentioned JP,6-309433,A arises. 

[0004]This invention is made in view of the above-mentioned siUiation. and the 
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contrast of the i^ce contained in 9 face picture is taken into consideration, A 
matcliing method and a device which can acquire the information for extracting a face 
area appropriately, It aims at providing the recording medium which recorded the 
program for making a computer perform this matching method, the image procBssing 
method using a device, a device, a matching method, and an image prooessing method 
and in which computer reading is possibis. 



[Trianslation done.] 
MEANS 



[Means for Solving the Prob(em]Whilo a matching method by this invention contains 
an outline of a face, and component parts of a face. Template matching which asks for 
a matching degree of several face templates which differ in the direction of a light 
souroe irradiated by direction of a kind of face and a face and face at least, and a face 
picture containing a person's face is performed, A face template with this highest 
matching degree is determined, and information about the direction of a light souroe 
irradiated by direction of a face and this face in a face area where ^ts matching 
destree contains a faco on said faoa picture based on the highest face template is 
acquired. 

[0006]Here, a "face template" contains component parts of faces, suoh as an outline 
of a face, eyes, a mouth, a nose, and an ear, about human being's face, and is obtained 
by changing at least the direction of a light source irradiated by direction of a kind of 
face and a face, and face. In a "face template" in this invention, when direction of a 
face changes, shape of an outline of a face differs from a locating position of 
component parts of a face. When the direction of a light source changes, 
concentration distribution of a field surrounded by outline of a face differs. For 
example, if the direction of a light source Is a transverse plana of a face, 
concentration distribution will become uniform, but if the direction of a light source Is 
right-hand side toward a template, it will be bright in right-hand side of a field 
surrounded by outline of a face, and left-hand side will serve as dark concentration 
distribution. As this "face template", a face may be expressed in three dimensions or 
it may express in two dimensions. As "a kind of face", an adult, a child, a woman, a 
male, yellow-skinned races, the white races, or a black race is mentioned. In addition 
to direction of a kind of face and a face, and the direction of a light souroe, a ftioe 
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template from which size of a face diffefs may be used, 

[0007]Dir8ction of a face can be known if a position of component parts of I^cbs, such 
as shape of an outilne of a face, eyes, and a nose, is l^nown. Therefore, "information 
about direotion of a face" means information about a position of contour shape of a 
face in a face templato with the highest matching degree, and component parts of a 
face. 

[0008]When the direction of a light source with which a face is irradiated is changed, 
distribution of a luminosity in a face will be changed variously and, as a result, 
concentration distribution of a face will be changed variously. Therefore, "information 
about the diraction of a light source" means information about oonoentration 
distribution of a face area in a face template with the highest matching degree. 
[00093In a matching method by this invention, it is preferred to set up a field of a 
prescribed range which includes said face area in said face picture as a face area 
oandidate, and to perform said template matching in this iace area candidate. 
[OOlOjHere, "a field of a prescribed range Including a face area" may be set up by 
displaying a face picture on a displaying means of a monitor etc., and surrounding a 
face area in fields, such as a rectangle and a round shape, and may be set up by 
dividing a face picture into two or more fields beforehand, and extracting a field 
including a face area from a divided field, 

[001 1]In a matching method by this Invention, said two or more face templates may be 
created beforehand. 

[001 2]lt creates by changing diroction of a face in two or more standard templates 
beforehand created in said two or more faoa templates according to a l<ind of said 
face, and the direction of a light source, It may be made to perform said template 
matching, acquiring a face template by which direction of this face and the direction of 
a light source were changed, 

[0013]In a matohing method by -Uifs Invention, it is preferred to perform said template 
matching by asking for a matohing degree hierarchical about said two or more face 
templates further again. 

[00l4]Hefe, "it asks for a matching degree hierarohicar fixes two elements (for 
example, direction of a face and the direction of a light source) among three elements 
of direction of a kind of face and a face, and the direction of a light source, makes one 
element (kind of face) a variable, and means performing template matching by 
changing only one element [ It ] [ it ] For example, when one element made into a 
variable is made into a kind of face, a kind efface is determined by performing 
template matching about a kind efface first, and asking for a template with the highest 
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matching degree. Subsequently, a kind of face and the difection of a face are 
determined by fixing a kind of face, perfor'ming template matching by making other on© 
alement (for example, direction of a face) into a variable, and asking for a template 
with iJiB highest matching degree. And a kind of fao© In a face picture, ihe direction of 
a face, and the direction of a light source are determined by fixing a kind of face, and 
direction of a face, performing template matching by making one element (the 
direction of a light source) of further others into a variable, and asking for a template 
with the highest matching degree. 

[001 5]Based on information about direction of said face and the direction of a light 
source which were acquired by a matching method by this invention, an image 
processing method by this invention. Beige concentration distribution in said face area 
is searched for, beige space in said face picture is defined based on this concentration 
distribution, and a field corresponding to said boiga space in said feoe picture is 
extracted as said face area. 

[001 elHare, saying * based on information about direction of a face and the direction 
of a light source, beige concentration distribution in a face area is searched for." An 
outline of a face area is presumed from. information about direction of a face, and it 
says acquiring distribution of a high-density area In an outline of a face area, and a 
low-density area from information about a position of component parts of a face in 
information about direction of a face, and the direotion of a light source. 
[001 7]It says [ "which defines beige space based on concentration distribution" ] 
appointing beige space at all the beige fields of a high-density area and a low-density 
area which exist all over a face area based on concentration distribution being 
included, 

[0018]In an image processing method by this invention, it is preferred to perform 
predateriviined image processing to said complexion range. 
[0019]Here, as "image processing", processing of versatility, such as a odor 
conversion process which considers a complexion range as a request and which is 
changed beige, gray-scale-conversion processing which changes gradation of a face 
area, and sharpness emphasis processing, can be used. 

[0020]This invention is characterized by a matching device comprising the following. 
While an outline of a face and component parts of a face are included, template 
matching which asks for a matching degree of several face templates whibh differ in 
the direction of a light source irradiated by direction of a kind efface and a face and 
face at least, and a face picture containing a person's face is performed, A template 
matching means which determines a face template with this highest matching degree, 
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An information acquisition means whioh aoquires infonnation about the direction of a 
iight source irradiated by direction of a face and this face in a face area where this . 
matching degree contains a face on said face picture based on the highest face 
template. 

[0021 ][n a matching device by this invention, It has further an area setting means 
which sets up a field of a prescribed range which includes said face area in said face 
picture as a face area candidate, and, as for said template matching means, it is 
preferred that it is a means to perform said template matching in said face area 
candidate. 

[0022]It may be made to have further a template memory measure which memorizes 
said two or more face templates in a matching device by this invention. 
[0023]In a matching device by this invention, A standard template memory measure 
which memorizes two or more standard templates beforehand created according to a 
kind of said face. It has further a template preparing means which oreates said two or 
more face templates by changing direction of a 'f^ce in said two or more standard 
templates, and the direction of a light source, It fs good also as a means to perform 
said template matching, acquiring a face template from which direction of said face 
created by this template preparing means and the direction of said light source differ 
said template matching means. 

[0024]As for said template matching means, in a matching device by this invention, it 
Is preferred that it Is a means to perform said template matching by asl^ing for a 
matching degree hierarchical about said two or more face templates further again. 
[0025]This invention Is characterized by an image processing device comprising the 
following. 

A means to search for beige concentration dls'bibution in said face area based on 
information about direction of said face and the direction of a light source which were 
acquired by a matching device by this invention. 

A means to define beige space in said face picture based on this concentration 
distribution. 

A means to extract a field corresponding to said beige space in said face picture as 
said face area. 

[0026]In an image processing device by this invention, it is preferred to have further a 
means to perform predetermined image processing to said complexion range. 
t0027]A matching matiiod and an image processing method by ^is invention may be 
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recorded on a recording medium in which computer reading is possible as a program 
for performlne: a computer, and may be provided. 



[Translation done.] 
DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] The block diagram showing^ the composition of the image processing 
device by the embodiment of this invention 

[Drawing 23 The figure for explaining the face area candidate's F specification 

[Drawing 3] The figure for explaining the face area candidate's F specification 

[Drawing 4] The figure showing the hierarchized template 

[Dravw'ng 5] The figure showing the example of a throo-dimensional template 

[Drawing 83T he figure for explaining exfe-action of a complexion range 

[Drawing 73T he figure showing the face area candidate who did Log-Polar conversion 

[Drawing SjT he flow chart which shows operation of this embodiment 

[Description of Notations] 

1 Reading means 

2 Face extraction means 

3 Image processing means 

4 Output means 

5 IVIonitor 

6 Input means 

1 1 Territorial extension means 

12 Area extraction means 

1 3 Template memory measure 

14 Mgtdhing means 

1 5 Complexion range extraction means 



[Translation done.] 
* NOTICES * 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

t.This document has been translated by computer. So the translation may not reflect 
the original precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



[Drawing 1] 
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[Drawing 2] 




[Drawing 3] 
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[Drawing 4] 
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[Drawjng 5] 
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[Drawing 8] 
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[Translation don©.] 
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